Papers by Zachary C. Lipton
Generating SOAP Notes from Doctor-Patient Conversations Using Modular Summarization Techniques (2021.acl-long)
Copied to clipboard
| Challenge: | Creating digital SOAP notes is burdensome and contributes to physician burnout . authors propose a pipeline to generate these notes based on transcripts of clinical conversations . |
| Approach: | They propose a pipeline to leverage deep summarization models based on conversations between physicians and patients . they propose an algorithm that extracts important utterances relevant to each section and generates one summary sentence per cluster . |
| Outcome: | The proposed algorithm outperforms its abstract counterpart by 8 ROUGE-1 points and produces more factual sentences as assessed by human evaluators. |
Does Pretraining for Summarization Require Knowledge Transfer? (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing theories claim that pretraining models learn linguistic knowledge from the pretraining corpus, but scientific explanations for these benefits remain unknown. |
| Approach: | They propose to use random character n-grams to test models on real corpora to see if the small residual benefit of using real data could be accounted for by the structure of the pretraining task. |
| Outcome: | The proposed task performs on documents consisting of character n-grams, whereas pretrained models perform on real corpora with no residual benefit. |
Weakly- and Semi-supervised Evidence Extraction (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to combine evidence annotations with document labels are limited to a minority of training examples. |
| Approach: | They propose to combine evidence annotations with abundant document labels for evidence extraction task. |
| Outcome: | The proposed method outperforms baselines on two classification tasks with evidence annotations. |
Combating Adversarial Misspellings with Robust Word Recognition (P19-1)
Copied to clipboard
| Challenge: | a sub-field of word recognition models is emerging to combat adversarial spelling mistakes . imperceptible attacks can cause models to misclassify examples, but training robust models remains a challenge . |
| Approach: | They propose to place a word recognition model in front of a downstream classifier to combat adversarial spelling mistakes. |
| Outcome: | The proposed model outperforms adversarial training and off-the-shelf spell checkers in a word recognition task. |
Learning to Deceive with Attention-Based Explanations (2020.acl-main)
Copied to clipboard
| Challenge: | Attention mechanisms are ubiquitous components in neural network architectures and are often claimed to confer interpretability. |
| Approach: | They propose a method for training models to produce deceptive attention masks by combining weights assigned to designated impermissible tokens with a weighted sum. |
| Outcome: | The proposed method reduces the weight assigned to designated impermissible tokens while still using them across multiple models and tasks. |
Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study (D18-1)
Copied to clipboard
| Challenge: | Existing studies on Active Learning (AL) for natural language processing have limited data requirements. |
| Approach: | They propose a Bayesian active learning approach that reduces deep learning's data dependence by comparing models and acquisition functions. |
| Outcome: | The proposed approach outperforms i.i.d. baselines and is more efficient than other approaches. |
Unsupervised Data Augmentation with Naive Augmentation and without Unlabeled Data (2021.emnlp-main)
Copied to clipboard
| Challenge: | Unsupervised Data Augmentation (UDA) is a semisupervised learning method that penalizes differences between a model's predictions on unlabeled examples and corresponding 'noised' examples produced via data augmentation. |
| Approach: | They propose to use a consistency loss to penalize differences between models' predictions on unlabeled and unlabed examples to enforce consistency between models and their perturbed counterparts. |
| Outcome: | The proposed method is able to penalize differences between models' outputs on unlabeled and unlabed examples without complex data augmentation. |
On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment (2020.emnlp-main)
Copied to clipboard
| Challenge: | Modern multilingual models are trained on concatenated text from multiple languages in hopes of conferring benefits to each (positive transfer) however, recent work has shown that this approach can degrade performance on high-resource languages, a phenomenon known as negative interference. |
| Approach: | They propose a meta-learning algorithm that adds language-specific parameters as meta-parameters and trains them in a manner that explicitly improves shared layers’ generalization on all languages. |
| Outcome: | The proposed model improves cross-lingual transferability and generalization on all languages, and improves on the language-specific parameters. |
Evaluating Explanations: How Much Do Explanations from the Teacher Aid Students? (2022.tacl-1)
Copied to clipboard
Danish Pruthi, Rachit Bansal, Bhuwan Dhingra, Livio Baldini Soares, Michael Collins, Zachary C. Lipton, Graham Neubig, William W. Cohen
| Challenge: | Existing methods to explain predictions by highlighting salient features are often unstated. |
| Approach: | They propose a framework to quantify the value of explanations via the accuracy gains that they confer on a student model trained to simulate a teacher model. |
| Outcome: | The proposed framework allows principled, automatic, model-agnostic evaluation of attributions. |
Entity Projection via Machine Translation for Cross-Lingual NER (D19-1)
Copied to clipboard
| Challenge: | a subset of languages have large annotated corpora for named entity recognition. |
| Approach: | They propose a system that leverages machine translation systems twice to improve named entity recognition. |
| Outcome: | The proposed system outperforms existing methods on Armenian languages by 4.1 points . it achieves state-of-the-art F_1 scores for Armenian, outperforming monolingual model trained on Armenia. |
How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks (D18-1)
Copied to clipboard
| Challenge: | Recent research addresses reading comprehension, where examples consist of (question, passage, answer) tuples. |
| Approach: | They establish sensible baselines for bAbI, SQuAD, CBT, CNN and Who-did-What datasets and compare them to their previous work. |
| Outcome: | The proposed models perform on 14 out of 20 bAbI, SQuAD, CBT, CNN and Who-did-What datasets. |
Practical Obstacles to Deploying Active Learning (D19-1)
Copied to clipboard
| Challenge: | Active learning (AL) is a widely-used training strategy for maximizing predictive performance subject to a fixed annotation budget. |
| Approach: | They propose to use active learning to optimize predictive performance . they find that current approaches do not generalize reliably across models and tasks . |
| Outcome: | The proposed approach outperforms training on i.i.d. datasets on supervised learning tasks. |
On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that adversarial data collection (ADC) models perform better on other adversarially collected data but are liable under plausible domain shifts. |
| Approach: | They conduct a large-scale controlled study on question answering by assigning workers at random to compose questions either adversarially (with a model in the loop) or in the standard fashion (without a modeling). |
| Outcome: | The proposed model performs better on other adversarial datasets but worse on diverse collection of out-of-domain evaluation sets. |